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I.  INTRODUCTION  AND  OBJECTIVE 


This  study  was  undertaken  to  evaluate  the  comparative  performance  of 
two  lightning  tracking  systems.  The  design  concept  of  both  systems  is 
based  upon  radio  direction  finding  (RDF)  at  separated  sites  and  triangulating 
lines  of  bearing  to  determine  the  location  of  the  signal  source.  The 
direction  finders  are  crossed  loop  sensors,  originally  proposed  by  Watson- 
Watt  and  Herd.(^)*  The  crossed  loop  direction  finder  is  well  known  for  its 
large  bearing  errors  in  the  presence  of  a  horizontally  polarized  electro¬ 
magnetic  field. (2)  (3)  To  overcome  this  difficulty,  both  systems  employ  a 
waveform  analysis  algorithm  which  is  designed  to  discriminate  between  cloud- 
to-ground  and  cloud-to-cloud  lightning  events.  Thus,  by  recognizing  the 
initial  portion  of  a  stroke  to  ground,  whirh  is  generally  oriented  vertically, 
horizontal  polarization  errors  can  be  reduced. 

The  objective  of  the  evaluation  is  to  assess  system  performance  in  terms 
of  four  criteria:  (1)  location  accuracy,  (2)  misidcntif ied  lightning  events, 
(3)  false  alarm  probability  and  (4)  failure  to  alarm  probability. 

Tests  were  performed  at  Kennedy  Space  Center.  Location  accuracy  was 
taken  from  two  sources  of  ground  truth:  (1)  a  network  of  three  all-sky 
cameras  deployed  by  SwRI  and  (2)  sightings  of  lightning  events  made  by  an 
observer.  The  evaluation  of  misidentif ied  lightning  events  is  determined 
by  the  incidence  of  cloud-to-cloud  flashes  classified  as  cloud-to-ground 
strikes.  The  false  alarm  probability  is  derived  from  the  observer  input 
and  weather  radar  data.  This  is  given  by  the  ratio  of  invalid  location 
reports  (for  which  there  is  no  meteorological  basis)  ,  to  the  total  number 
of  reports  by  each  system.  The  f ailure-to-alarm  probability  is  the  ratio 
of  missed  cloud-to-ground  strikes,  identified  by  the  camera  network  or 
observer,  to  the  total  number  of  positively  identified  cloud-to-ground 
strikes. 


*See  References. 
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II.  EXPERIMENTAL  DESIGN 


The  experimental  design  of  this  study  was  dictated  in  part  by  the 
evaluation  of  six  lightning  warning  systems  for  the  U.  S.  Bureau  of  Mines 
under  Contract  J0387207.  The  evaluation  reported  herein  was  undertaken  as 
a  supplementary  effort  to  the  work  funded  by  the  Bureau  of  Mines. 

A.  System  Operation 

The  direction  finding  sensors  for  both  systems  are  crossed  loop 
antennas  whose  principle  of  operation  has  been  exploited  since  the  1920’ s. 

The  crossed  loop  method  of  direction  finding  has  a  180°  ambiguity  in  deter¬ 
mining  the  azimuthal  angle  of  arrival.  This  problem  is  resolved  differently 
in  the  two  systems.  The  Lightning  Location  and  Protection  (LLP)  system  uses 
a  flat  plate  electric  field  sensor  at  each  crossed  loop  site  to  determine 
whether  the  lightning  discharge  lowered  positive  or  negative  charge  to  earth. 

A  positive  electric  field  excursion  implies  that  negative  charge  was  lowered, 
and  thus  the  direction  of  the  magnetic  field  can  be  determined.  This  resolves 
the  180°  ambiguity. 


The  bearing  ambiguity  is  resolved  somewhat  differently  in  the  Atlantic 
Scientific  Corporation  (ASC)  system.  In  this  system,  three  crossed  loop 
sensors  are  deployed  and  the  two  lines  of  bearing  from  each  site  are  projected 
to  compute  location  predictions.  The  location  which  best  satisfies  a 
"consistency"  criterion  is  determined  to  be  the  correctly  resolved  positional 
fix,  the  optimally  consistent  fix  being  a  single  point  where  all  three  lines 
of  bearing  intersect. 


As  pointed  out  by  Taylor^2)  and  Uman,  et  al,(3)  crossed  loop  direction 
finding  errors  due  to  horizontal  polarization  can  be  extremely  large  for 
signal  sources  in  close  proximity  to  the  antenna  site.  To  overcome  errors 
due  to  horizontal  components,  both  the  LLP  and  ASC  systems  employ  a  signal 
recognition  algorithm  which  discriminates  cloud-to-ground  versus  inter-cloud 
lightning  events.  The  cloud-to-ground  test  is  based  on  two  criteria:  (1)  time 
to  initial  peak,  typically  5  to  10  microseconds,  and  (2)  decay  time,  approxi¬ 
mately  40-50  microseconds.  If  these  conditions  are  satisfied,  the  initial 
peak  value  is  used  to  determine  the  angle  of  arrival.  A  detailed  description 
of  this  technique  is  given  by  Krider,  et  al(^)  and  Herrman,  et  al,^-''  Only 
the  initial  portion  of  the  received  waveform  is  used  for  direction  finding 
analysis  (typically  the  first  100  meters  of  the  lightning  channel).  Thus,  it 
is  assumed  that  this  portion  of  the  return  stroke  is  predominately  oriented 
vertically. 


Although  the  concept  of  operation  is  the  same  in  both  systems,  there  are 
significant  differences  in  the  implementation.  In  the  LLP  system  the  line  of 
bearing  to  a  lightning  strike  is  determined  at  each  of  the  two  remote  sites. 
The  bearing  is  computed  to  each  return  stroke  (maximum  of  8)  in  the  lightning 
flash.  These  are  tested  for  consistency  and  averaged.  The  results  of  this 
analysis  is  then  transmitted  to  a  position  analyzer  at  the  central  facility. 
Given  that  the  reports  from  the  two  remote  sites  are  time  coincident,  the 
position  analyzer  computes  a  location  fix  at  the  point  where  the  lines  of 


m 
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bearing  intersect.  Communication  between  the  two  remote  sites  and  the 
central  facility  is  half  duplex  over  single  pair  land  line  wires  at  300 
baud  to  each  site.  A  transmitting  modem  is  required  at  each  remote  site 
and  two  receiving  modems  are  required  at  the  central  facility. 

The  ASC  system  uses  three  crossed  loop  sensors  as  compared  to  the  two 
antenna  deployment  of  the  LLP  system.  In  the  ASC  implementation,  the  received 
signals  from  the  lightning  strikes  are  digitized  at  each  of  the  three  sites 
and  the  digital  data  are  transmitted  to  the  central  facility.  The  central 
processor  performs  the  waveform  analysis,  computes  lines  of  bearing  from  the 
three  sites  and  calculates  the  location  of  the  lightning  strike.  The  ASC 
system  does  not  compute  a  bearing  on  the  initial  return  stroke  in  the  flash, 
since  the  manufacturer  believes  that  the  branching  which  frequently  occurs 
in  this  stroke  (and  not  in  subsequent  strokes)  produces  sufficient  horizontally 
polarized  components  to  vitiate  a  bearing  measurement.  It  is  observed  that 
this  algorithm  does  not  permit  location  estimates  of  single  stroke  lightning 
flashes.  Communication  from  the  three  remote  sites  to  the  central  facility 
is  full  duplex  over  two  pairs  of  land  line  wires  at  1200  baud. 

An  additional  difference  in  the  analysis  technique  used  in  the  two 
systems  is  the  signal  passband  of  the  remote  sites.  The  LLP  passband  spans 
approximately  1  kHz  to  400  kHz  while  the  ASC  system  passband  is  1  kHz  to 
50  MHz.  Thus,  both  systems  employ  wideband  receivers  and  signal  processing. 

The  bandwidths  reported  here  are  those  expressed  verbally  by  each  manufacturer 
and  are  assumed  to  be  approximate. 

The  concept  of  gating  a  receiver  on  the  initial  portion  of  a  received 
atmospheric  to  reduce  bearing  errors  due  to  horizontal  polarization  is  not  a 
new  technique  but  has  been  employed  for  a  number  of  years. ^  The  new 
technology  being  exploited  in  these  systems  is  the  discrimination  of  cloud- 
to-ground  lightning  events  versus  inter-cloud  discharges. 

B.  Optical  Location  System 

To  provide  precise  lightning  location  data  for  the  assessment  of 
positional  accuracy  of  the  two  systems,  a  network  of  three  all-sky  cameras 
was  constructed.*  One  of  the  all-sky  cameras  is  shown  in  Figure  1.  The 
antenna  system,  stand,  and  battery  power  are  shown  in  Figure  1(a).  Cloud-to- 
ground  lightning  events  were  detected  at  the  SwRI  mobile  laboratory.  The 
ground  flash  detector  was  a  modified  version  of  a  flash  counter  described 
by  Prentice,  et  al.^?)  When  a  ground  strike  was  detected,  the  circuit  keyed 
a  235  MHz  transmitter.  This  signal  was  received  at  the  camera  sites  and  was 
used  to  actuate  the  camera  shutter.  The  VHF  radio  receivers  at  each  of  the 
three  sites,  transmitting  monopoles,  and  11-element  directional  receiving 
antennas  were  acquired  as  stock  commercial  equipments. 

As  shown  in  Figure  1(a)  the  camera  and  associated  electronics  are 
powered  by  the  12-volt  storage  battery  located  beneath  the  stand.  The 


♦Funded  under  another  contract  in  this  laboratory. 


(a)  Receiving  Antenna  and  Housing 
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camera  views  a  parabolic  reflector  which  images  the  visible  sky  to  the 
horizon.  The  camera  shutters  are  activated  by  the  initial  lightning  stroke 
with  an  inherent  delay  time  of  95  milliseconds. 

The  close-up  view  in  Figure  1(b)  illustrates  the  access  to  the  camera 
and  receiver  electronics  located  in  the  upper  portion  of  the  stand.  When 
the  hinged  door  is  closed,  the  camera  looks  down  upon  the  convex  parabolic 
reflector  and  a  clock  as  shown  in  Figure  1(c).  A  lightning  strike  is  shown 
in  Figure  1(c)  illustrating  the  triangulation  algorithm.  Each  camera  was 
surveyed  so  that  0  degree  on  the  template  is  aligned  with  true  north.  In 
the  case  shown,  lightning  struck  ground  at  an  azimuth  of  345.5  degrees 
from  the  site.  This  procedure  was  originally  employed  by  Kidder. « ) 

Also  shown  in  Figure  1(c)  are  the  LEDs  used  for  nighttime  photography 
situated  at  0,  90,  180  and  270  degrees  with  two  at  0  degree.  The  LEDs  are 
used  to  align  an  azimuth  template  for  night  photographs  since  the  azimuthal 
grid  does  not  photograph  in  the  dark. 

C.  Visual  Observations 


During  the  period  of  evaluation,  an  observer  was  provided  by  the  Cape 
Canaveral  Air  Force  Station.  The  observer  reported  lightning  events  which 
were  visible  from  the  cupola  atop  the  O&C  Building  at  Kennedy  Space  Center. 
This  vantage  point  provided  a  clear  view  of  the  NASA  complex,  although  there 
was  some  degree  of  visual  obstruction  toward  the  east  southeast. 

Shown  in  Figure  2  is  the  format  of  the  visual  lightning  events  reported 
by  the  observer.  The  report  provides  time  resolved  to  the  nearest  second, 
nominal  direction  to  the  lightning  event,  whether  the  strike  was  to  ground 
or  inter-cloud  and  space  for  remarks  concerning  unusual  phenomena.  The  time 
standard  used  by  the  observer  and  those  used  in  both  location  systems  were 
synchronized  to  WWV.  Thus,  worst  case  error  in  time  synchronization  was  of 
the  order  of  one  second.  A  compass  had  been  mounted  in  the  cupola  so  that 
nominal  direction  to  the  strike  could  be  determined  within  +  10  degrees. 

Additional  support  furnished  by  the  Air  Weather  Service  at  Cape  Canaveral 
included  storm  alert  notices.  SwRI  was  notified  when  thunderstorm  activity 
was  indicated  within  a  25  nautical  mile  radius  of  the  central  facility. 

Also  a  direct  telephone  line  was  connected  between  the  Air  Weather  Service 
radar  facility  and  the  SwRI  mobile  laboratory  to  provide  storm  tracking 
information  while  data  acquisition  was  in  progress. 

D.  Location  System  Deployment 

The  deployment  of  system  antennas  is  shown  in  Figure  3.  The  LLP  antennas 
were  located  at  DF  site  No.  1  (block  house  43)  and  DF  site  No.  2  (slide  wire 
site  near  pad  39A) .  Two  of  the  ASC  antennas  were  co-located  with  the  LLP 
sensors  and  the  third  antenna  was  located  at  DF  site  No.  3  (Tico  Airport). 

Also  shown  in  Figure  3  is  the  placement  of  the  SwRI  mobile  laboratory  at 
camera  site  FCA1  and  the  deployment  of  the  all-sky  cameras  at  camera  site 
1-3. 
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The  SwRI  mobile  laboratory  is  illustrated  in  Figure  4(a).  The 
laboratory  is  a  23-foot  aluminum  trailer.  Figure  4(b)  shows  an  interior  view 
of  the  mobile  laboratory  in  which  the  central  processor  for  both  systems  were 
situated . 

Shown  in  Figure  5  is  the  sensor  deployment  for  the  LLP  antennas. 

Figure  5(a)  shows  DF  site  No.  2  near  pad  39A,  and  Figure  5(b)  shows  DF  site 
No.  1  near  block  house  43.  The  deployment  of  the  ASC  antennas  are  shown  in 
Figure  6.  The  antenna  situated  at  Tico  Airport  is  shown  in  Figure  6(a).  The 
sensor  at  block  house  43  is  illustrated  in  Figure  6(b)  and  the  antenna  placed 
near  pad  39A  is  shown  in  Figure  6(c).  It  is  observed  that  at  the  latter 
two  sites,  the  sensors  were  mounted  atop  utility  poles. 

Deployment  sites  for  the  optical  location  system  is  shown  in  Figure  7. 

The  camera  site  atop  the  O&C  Building  is  illustrated  in  Figure  7(a).  The 
camera  site  near  pad  39A  is  shown  in  Figure  7(b)  and  the  site  atop  the  pass 
and  ID  Building  is  displayed  in  Figure  7(c).  It  had  been  planned  to  situate 
a  camera  system  near  gate  three  at  the  western  entrance  to  Kennedy  Space 
Center;  however,  this  site  was  abandoned  in  favor  of  the  O&C  Building  to 
increase  the  probability  of  detecting  grounded  strikes  by  more  than  one 
camera  system. 

The  location  of  the  SwRI  mobile  laboratory  is  shown  in  Figure  8.  As 
can  be  observed  from  the  illustration,  the  elevated  camera  site  provided 
an  unobstructed  view  in  all  directions  which  enabled  operators  to  visually 
track  thunderstorm  activity  in  the  area. 

The  electronics  associated  with  each  system  are  shown  in  Figure  9.  The 
processors  at  the  remote  sites  are  illustrated  in  Figure  9(a).  The  large 
chassis  situated  in  the  left  most  portion  of  the  shelf  is  the  ASC  processor 
and  the  two  smaller  chassis  (stacked  one  upon  the  other) ;at  the  right  of  the 
shelf  are  the  LLP  processors.  The  devices  between  the  two  are  telephone 
modems.  The  ASC  electronics  located  at  the  SwRI  mobile  laboratory  is  shown 
in  Figure  9(b).  This  includes  a  line  printer,  a  floppy  disk  drive,  a  central 
processor  and  a  display  console.  The  LLP  electronics  are  shown  in  Figure  9(c) 
and  includes  a  plotter,  a  central  processor  and  a  TTY. 

E.  Installation  and  Data  Acquisition 

On  3  May  1979,  Southern  Bell  Telephone  Company  was  requested  by  SwRI  to 
install  data  modems  connecting  the  remote  sites  to  the  central  processors  in 
the  SwRI  mobile  laboratory.  This  work  was  to  be  completed  on  or  before 
31  May  so  that  tests  could  begin  on  1  June  1979  and  conclude  on  13  July  1979. 
The  SwRI  mobile  laboratory  was  placed  on  site  24  May  1979  and  departed 
Kennedy  Space  Center  on  13  July  1979.  Southern  Bell  indicated  that  modem 
installation  was  scheduled  to  begin  on  29  May  1979  and  complete  by  1  June 
1979.  Actual  completion  date  of  installation  by  Southern  Bell  was  15  June 
1979.  Deployment  by  ASC  commenced  on  14  June  1979  and  was  completed  on 
26  June  1979.  The  LLP  system  was  installed  and  operational  on  15  June  1979; 
however,  a  failure  in  the  telephone  modems  was  observed  on  16  lune  1979 
and  corrected  on  21  June  1979. 
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An  initial  task  in  the  conduct  of  this  evaluation  was  the  verification 
of  performance  of  the  ASC  location  system.  During  the  afternoon  of  26  June 
1979,  1700  EDT,  an  isolated  convective  cell  developed  within  a  triangular 
area  bounded  by  Melbourne,  St.  Cloud  and  Cocoa  Beach,  Florida.  This  cell 
was  located  22  to  25  miles  distant  from  the  SwRI  mobile  laboratory  and  was 
the  only  discernible  echo  on  the  Cape  Canaveral  Air  Force  Station  Air  Weather 
Service  radar.  The  data  collected  during  this  period  are  shown  in  Figure  10. 

The  cross-hair  plot  on  the  NWS  radar  map  are  the  locations  computed  by  the 
LLP  system.  The  inset  map  at  the  upper  right  is  the  data  plot  photographed 
on  the  ASC  display  CRT.  The  dots  indicate  lightning  strike  locations.  The 
cluster  of  lightning  activity  as  detected  by  both  systems  was  in  agreement 
with  the  meteorological  activity  indicated  by  weather  radar.  The  lightning 
strikes  were  occurring  at  the  rate  of  one  every  ten  to  twenty  seconds,  and 
it  was  observed  that  both  systems  were  reporting  ground  strike  locations  on 
the  same  flash,  the  LLP  system  computing  somewhat  faster.  The  ASC  operating 
software  at  this  time  did  not  report  individual  flash  location  data  to  the 
line  printer.  This  required  a  subsequent  modification  to  provide  the 
capability  for  comparison  testing  with  the  LLP  system.  Based  upon  the 
observations  made  during  this  single  storm  event,  it  was  determined  that  the 
ASC  system  had  the  capability  to  detect  and  track  lightning  events. 

During  the  period  27  June  1979  through  9  July  1979,  the  ASC  system  was 
undergoing  software  development  to  incorporate  line  printer  output.  Updated 
software  implementing  the  line  printer  output  was  operational  on  8  July. 
Additional  enhancements  provided  were  refinements  to  the  CRT  map  display  and 
a  circle  drawn  about  developing  or  active  storm  cells  as  determined  from  the 
computed  lightning  location  data.  The  modified  display  is  shown  in  Figure  11; 
however,  there  exists  no  corresponding  line  printer  output  for  these  data. 

On  8  July  1979  a  power  interrupt  caused  the  LLP  processor  to  restart 
with  the  internal  clock  locked  in  a  fixed  state.  This  was  due  to  weakened 
batteries  which  failed  to  maintain  clock  operation  during  power  outage.  The 
LLP  system  was  operated  on  8  and  9  July  in  this  mode.  The  condition  was  not 
catastrophic  since  the  remote  site  reported  a  line  of  bearing  to  the  lightning 
strike.  A  computer  program  was  written  to  postprocess  the  data  for  a  deter¬ 
mination  of  lightning  strike  locations.  On  10  July  1979  GMT,  the  clock  was 
reset  at  the  remote  site.  The  impact  of  the  clock  problem  is  that  the  LLP 
system  could  not  locate  lightning  flashes  which  were  separated  by  fewer  than 
eight  seconds  in  real  time.  Thus,  it  is  possible  that  some  lightning  events 
were  missed  during  operation  on  8  and  9  July. 

The  data  acquisition  period  over  which  both  systems  were  operational  was 
8  July  1979  1633-1808  GMT,  9  July  1979  0647-0719  GMT,  10  July  1979  0130-0330  GMT, 
and  10  July  1979  1955-2045  GMT. 


Performance  Verification 


III.  DATA  ANALYSIS 


A.  Optical  Locator  Performance 

The  three  all-sky  locator  cameras  did  not  provide  the  consistent  lightning 
location  data  anticipated  for  the  tests.  After  the  initial  development  and 
success  with  one  camera  at  the  SwRI  site,  it  was  expected  that  the  system 
would  produce  useful  location  results  routinely  whenever  electrical  storms 
were  within  viewing  range  of  at  least  two  cameras. 

Based  on  primarily  mid  to  late  afternoon  storm  activity  at  KSC,  neutral 
density  filters  were  acquired  to  overcome  problems  of  daytime/nighttime 
exposures.  In  addition,  separate  development,  mixtures  for  daytime  and  for 
nighttime  exposures  provided  twenty  four-hour  capability.  Daytime  and 
nighttime  photographs  taken  at  KSC  demonstrated  this  approach  was  viable. 

During  testing  at  KSC,  however,  the  inexpensive,  relatively  broadband 
camera  trigger  receivers  were  found  to  be  vulnerable  to  the  high  density 
interfering  signal  environment  at  KSC.  Consequently,  the  camera  system 
continually  false  triggered.  As  a  result  of  variable  and  persistent  inter¬ 
ference,  only  four  two-camera  correlated  lightning  photographs  were  obtained 
during  the  six  week  test  period  at  KSC.  Moreover,  none  of  these  data  coincided 
in  time  with  the  period  of  time  in  which  both  location  systems  were  operational. 

Although  use  of  an  optical  lightning  location  system  is  a  good  approach, 
experience  in  these  tests  shows  improved  camera  trigger  receivers  with  signifi¬ 
cantly  less  vulnerability  to  radio  frequency  interference  are  required.  In 
addition,  at  least  two  more  cameras  should  be  deployed  in  the  network  to 
enhance  the  probability  of  lightning  strike  location  for  baselines  of  the 
order  of  seven  to  ten  miles  in  the  presence  of  overcast  and  obscurity  due  to 
rain  encountered  under  field  conditions. 

B.  Storm  of  8  July  1979  (1633-1808  GMT) 

At  midday  local  time,  a  storm  system  developed  over  the  KSC  complex  and 
extended  approximately  100  nmi  southward.  The  storm  is  shown  in  Figure  12 
as  observed  on  the  NWS  radar  at  Daytona  Beach  at  approximately  1638  GMT. 

Figure  13  displays  the  system  an  hour  later  at  1732  GMT  and  Figure  14  was 
taken  at  1808  GMT.  As  is  indicated  in  the  sequence  of  radar  pictures,  the 
storm  was  stationary  over  its  lifetime. 

During  the  storm,  the  ASC  system  reported  95  lightning  events.  These  are 
plotted  in  Figure  15.  A  comparison  of  the  data  with  the  meteorological  reports 
indicate  a  reasonable  degree  of  correlation.  One  feature  of  concern  is  noted. 
The  scatter  of  points  to  the  northwest  and  to  the  southwest  extremes  lie 
on  a  straight  line.  It  would  appear  that  in  many  cases  the  east-west  value 
remained  fixed  while  only  the  north-south  location  estimate  varied.  It 
seems  to  be  a  highly  unlikely  distribution  of  lightning  strikes;  however, 
the  median  scatter  about  the  west  is  quite  reasonable. 
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Figure  16  shows  the  corresponding  scatter  plot  for  the  LLP  system.  During 
this  same  time  period  64  lightning  events  were  reported  by  the  LLP  device.  The 
primary  reason  for  the  difference  in  lightning  event  counts  resides  in  the 
fact  that  the  LLP  could  report  lightning  events  no  closer  in  time  than  eight 
seconds.  In  this  operating  mode,  a  review  of  the  data  output  indicates  that 
the  buffer  capacity  was  sufficient  to  facilitate  four  observations  from  each 
remote  site.  These  data  were  then  output  pairwise  to  the  line  printer.  Time 
was  assigned  on  output  so  that  multiple  return  strokes  were  separated  by 
eight  seconds.  Thus,  in  a  stiict  sense,  time  of  the  lightning  strike  had 
several  seconds  of  variability.  Presumably,  data  were  lost  while  multiple 
stroke  reports  were  being  made  and  worst  case  would  have  been  32  seconds. 

Again,  the  data  seem  to  be  scattered  in  reasonable  agreement  with  the 
meteorological  data. 

During  this  period  of  time,  12  cloud-to-ground  strikes  were  reported  by 
the  observer  in  the  O&C  Building.  These  were  located  to  the  west  and  northwest. 
The  visual  sightings  would  tend  to  correlate  most  favorably  with  the  ASC 
scatter  shown  in  Figure  15;  however,  not  one  visually  reported  ground  strike 
can  be  credibly  correlated  with  a  location  report  made  by  either  the  ASC  or 
LLP  system.  Correlations  were  attempted  taking  into  account  the  differences 
in  reporting  time,  computational  time,  systematic  time  offsets,  etc.,  and 
nothing  yielded  credible  results. 

Correlation  of  lightning  events  reported  by  the  two  systems  was  also 
attempted  and  again  no  degree  of  correlation  was  evident.  This  was  not  a 
surprising  result  in  view  of  the  obvious  differences  in  the  scatter  patterns 
of  Figures  15  and  16. 

C.  Storm  of  9  July  1979  (0647-0719  GMT) 

Shown  in  Figure  17  is  the  NWS  radar  trace  of  a  large  storm  system  at 
0658  GMT.  Although  widespread,  this  proved  to  be  a  relatively  short  duration 
storm  of  moderate  electrical  activity. 

The  scatter  plot  of  lightning  events  reported  by  the  ASC  system  is  shown 
in  Figure  18.  There  were  24  reported  locations.  The  row  structure  of 
location  data  which  were  observed  in  the  storm  of  8  July  is  evidenced  here 
in  the  east-west  direction.  In  this  case  a  number  of  points  appear  fixed  in 
the  north-south  direction  and  vary  only  in  the  east-west  direction. 

A  corresponding  scatter  plot  for  the  LLP  system  is  shown  in  Figure  19. 

This  plot  consists  of  30  points  which  would  suggest  that  the  interval  between 
strikes  was  longer  than  for  the  storm  of  8  July,  and  thus,  the  number  of  events 
reported  by  the  two  systems  are  in  closer  agreement.  A  review  of  the  times  of 
the  reported  strikes  reveals  that  this  was  in  fact  the  case. 

Nine  visual  observations  of  cloud-to-ground  strikes  were  made  during  this 
storm.  They  were  directed  toward  the  northeast  which  would  suggest  a  high 
degree  of  correlation  with  the  LLP  data  shown  in  Figure  19.  Of  the  nine  visual 
reports  only  one  appeared  to  be  in  agreement  with  the  LLP  data  and  none  were 
found  to  correlate  with  the  ASC  data. 


FIGURE  16.  LLP  LIGHTNING  LOCATIONS,  8  JULY  1979  1633-1840  GMT 
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Again  an  attempt  was  made  to  correlate  the  location  reports  made  by  the 
two  systems  and  no  credible  correlation  was  evident.  As  was  pointed  out 
previously,  a  visual  comparison  of  the  two  scatter  plots  would  suggest  that 
no  correlation  existed. 


Storm  of  10  July  1979  (0130-0330  GMT) 


During  the  late  evening  at  9:30  p.m.,  9  July  local  time  a  severe  thunder¬ 
storm  moved  inland  from  the  Atlantic  ocean  and  produced  a  tornado  which  caused 
considerable  property  damage  in  the  Cocoa  Beach  area.  The  storm  was  extremely 
active  electrically. 


Figure  20  is  a  scatter  plot  of  the  ASC  data  reported  during  this  storm. 
Ninety  location  computations  were  indicated.  As  is  shown  in  the  plot,  the 
data  show  a  heavy  concentration  in  the  Cocoa  Beach  area  in  association  with 
the  most  intense  meteorological  activity. 


Figure  21  is  a  plot  produced  by  the  LLP  system  during  this  time.  The  LLP 
clock  at  the  remote  site  had  been  reset  so  that  the  system  was  fully  operational. 
The  LLP  system  reported  689  lightning  events  within  this  time  span. 


During  this  period  there  were  25  visually  observed  cloud-to-ground  strikes. 
Correlating  the  LLP  data  with  the  visual  data  it  was  determined  that  the  LLP 
clock  was  two  seconds  fast  with  respect  to  the  observer's  clock.  Using  time 
and  azimuth  as  reported  by  the  observer,  21  lightning  location  events  were 
found  to  be  highly  correlated  which  would  suggest  that  the  LLP  system  has  a 
detection  probability  of  the  order  of  84  percent.  In  the  case  of  the  ASC 
data  it  was  found  the  clock  was  five  seconds  slow  with  respect  to  the  observer's 
clock.  Of  the  lightning  events  reported  by  the  ASC  system,  14  locations  could 
be  reasonably  correlated  with  the  visual  data  which  implies  a  detection  prob¬ 
ability  of  56  percent. 


It  was  also  found  that  the  mean  time  to  report  a  location  following  a 
visual  sighting  was  1.76  seconds  for  the  LLP  system  and  6.86  seconds  for  the 
ASC  system.  This  no  doubt  contributed  significantly  to  the  large  differential 
in  reported  locations  between  the  two  systems. 


A  correlation  was  made  between  the  time  and  location  reports  output  by 
both  systems.  In  this  correlation  it  was  found  that  the  ASC  clock  was  seven 
seconds  slow  compared  to  the  LLP  clock.  This  is  in  agreement  with  the  earlier 
time  slip  found  between  each  system  individually  and  the  observer's  clock. 

Forty  five  points  were  determined  to  be  reasonably  correlated.  A  plot  of  these 
data  are  shown  in  Figure  22.  Both  systems  appear  to  be  in  agreement  that  the 
most  intense  activity  is  in  the  vicinity  of  Cocoa  Beach. 


A  system  of  storms  developed  during  the  late  afternoon,  4:00  p.m.  local 
time,  on  10  July.  These  are  shown  in  the  radar  trace  of  Figure  23.  These 
storms  were  of  moderate  to  light  in  electrical  intensity. 


LIGHTNING  LOCATIONS,  10  JULY  1979  0130-0330  GMT 


Figure  2 1 .  LLP  Lightning  Locations,  10  July  79,  1408  SMBS  P". 77  ;  .  ''  '■  :  ,;,TIY  F  ivACI 
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FIGURE  22.  ASC  ond  LLP  CORRELATED  LIGHTNING  LOCATIONS,  10  JULY  1979  0130-0330  GMT 
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A  composite  plot  showing  the  ASC  and  LLP  data  are  given  in  Figure  24. 

The  ASC  reported  15  locations  some  75  km  north-northwest  of  the  test  site. 

The  LLP  data  show  scatter  to  the  southwest  and  to  the  northwest  with  only 
one  point  of  agreement  with  the  ASC  data.  The  LLP  system  reported  36 
locations . 

It  is  believed  that  both  systems  were  fully  operational  during  this 
storm  so  that  the  effects  of  impaired  operation  evidenced  on  8  and  9  July  are 
not  contributing  to  the  differences  observed  here.  It  is  believed  that  the 
ASC  system  reported  no  location  results  in  agreement  with  the  LLP  system 
because  of  system  implementation.  At  no  time  during  the  test  did  the  ASC 
system  report  locations  beyond  a  range  of  150  km.  It  is  not  known  whether 
this  is  a  software  or  hardware  design  decision.  It  is  a  matter  of  speculation 
why  the  LLP  system  did  not  report  locations  in  agreement  with  the  ASC  system 
(with  the  exception  of  one  point).  A  number  of  possible  reasons  exist,  however, 
in  the  absence  of  ground  truth  data  no  determination  can  be  made. 

No  visual  data  were  obtained  for  cloud-to-ground  strikes  during  this 
final  test  period. 
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IV.  DISCUSSION 


One  major  feature  of  this  test  is  the  small  quantity  of  comparative 
performance  data  obtained  during  the  six  week  test  period.  One  contributing 
factor  was  the  two  week  delay  by  the  Southern  Bell  Telephone  Company  in 
completing  installation  of  the  communication  modems.* 

The  LLP  system  was  installed  in  parallel  with  the  modem  installation. 

The  system  was  operational  for  one  day,  15  June;  however,  a  telephone  modem 
failure  prevented  data  acquisition  until  the  problem  was  corrected  by  Southern 
Bell  on  21  June  1979.  This  proved  not  to  be  a  significant  factor  since  only 
four  thunderstorms  occurred  during  the  first  three  weeks  of  test.  Somewhat 
more  inhibiting  was  the  failure  of  the  LLP  remote  site  to  restart  after  a 
power  interrupt.  During  power  outage,  the  batteries  were  too  weak  to  maintain 
operation  of  the  clock,  and  a  backup  mode  of  data  acquisition  was  initiated  on 
8  and  9  July.  As  was  pointed  out  earlier,  system  performance  appears  to  have 
been  somewhat  impaired  during  this  period. 

Installation  of  the  ASC  system  was  begun  on  14  June  and  completed  on 
26  June  1979,  the  delay  being  due  primarily  to  the  difficulty  in  obtaining 
"cherry  pickers"  for  the  mounting  of  antennas  atop  telephone  poles.  Although 
a  performance  checkout  of  the  ASC  system  was  accomplished  on  26  June,  the 
system  software  did  not  provide  hard  copy  output  to  line  printer  as  had  been 
requested.  ASC  system  software  modification  and  hardware  adjustment  was  not 
completed  until  the  evening  of  9  July  local  time  (10  July  GMT) .  The  ASC 
system  was  never  operated  in  an  automatic  unattended  data  acquisition  mode. 
During  the  course  of  the  evaluation,  a  representative  from  ASC  was  required 
to  be  in  attendance  to  operate  the  system.  Diskettes  were  left  for  SwRT 
personnel  to  load  and  initiate  data  acquisition;  however,  in  everv  case  the 
system  failed  to  operate  when  loaded  from  these  diskettes. 

One  of  the  parameters  which  was  to  be  used  in  the  assessment  of  system 
performance  was  the  incidence  of  misidentif ication  of  inter-cloud  lightning 
as  cloud-to-ground  strikes.  It  was  determined  that  this  task  was  not  feasible 
using  single  point  observations.  The  problem  which  frequently  arises  is  that 
a  glow  in  the  cloud  is  visually  identified  as  inter-cloud  lightning  when  in 
fact  a  ground  strike  is  obscureu  by  precipitation  and  the  observer  can  only 
see  the  streamer  processes  associated  with  the  ground  strike.  A  network 
of  observers  should  be  deployed  to  overcome  these  problems  and  obtain  a 
credible  assessment  of  misidentif ied  lightning  events. 

Another  parameter  which  was  to  be  used  in  evaluating  system  performance 
was  location  accuracy.  These  data  were  to  be  provided  by  the  network  of 
all-sky  cameras  whose  performance  has  already  been  discussed.  One  interesting 
aspect  of  location  results  is  in  the  comparison  of  data  between  the  two 


*Similar  experience  in  San  Antonio  had  indicated  that  the  modems  could  be 
installed  in  three  to  four  weeks  after  the  order  had  been  placed,  however, 
twice  that  time  was  required  at  KSC. 
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systems  from  the  tornadic  storm  of  10  July.  It  is  recalled  that  there 
appeared  to  be  45  lightning  events  which  could  be  correlated  in  time.  Of 
these  simultaneous  events  only  seven  locations  agree  within  five  percent 
or  less  as  a  function  of  range.  This  can  be  observed  from  the  scatter  plot 
of  Figure  22  where  few  points  are  directly  superposed.  A  factor 
which  contributed  to  the  differences  between  corresponding  location 
estimates  was  a  500  meter  error  in  fixing  t he  position  of  DF  site  No,  1. 
This  error  was  corrected  in  the  ASC  system;  however,  in  the  LLP  system  the 
correction  was  considere'1  to  be  inconsequential  for  the  Bureau  of  Mines 
evaluation.  Had  sufficient  ground  truth  locations  been  obtained,  the  LLP 
locations  could  have  been  postprocessed  to  make  this  correction. 

A  third  parameter  which  was  examined  in  this  evaluation  was  the  false 
alarm  rate,  that  is,  those  cases  of  reported  lightning  for  which  there  was 
no  credible  meteorological  activity.  Neither  system  appeared  to  estimate 
lightning  events  where  there  existed  no  possibility  of  electrical  acLivity. 

In  summary,  while  the  initial  objectives  of  the  evaluation  were 
incompletely  achieved,  the  test  did  produce  sufficient  data  to  support  a 
partial  assessment  of  the  comparative  performance  of  the  two  systems. 

The  three  site  (ASC)  antenna  system  deployment  offers  a  potentially 
higher  degree  of  location  accuracy  than  the  two  site  (LLP)  deployment. 

The  theory  of  triangulation  nets  clearly  indicates  the  superiority  in  fix 
reliability  of  a  three  station  net  compared  to  a  two  station  net.  One 
tradeoff  is  the  additional  cost  of  the  third  land  line  for  communication; 
however,  the  improvement  in  location  accuracy  may  be  cost  effective. 

Both  systems  require  a  laboratory  environment  for  the  electronics  at 
the  remote  sites.  Since  both  systems  can  tolerate  only  100  feet  of  lead 
length  to  the  antennas,  this  means  that  the  crossed  loop  direction  finders 
may  be  subjected  to  significant  site  errors  due  to  reradiation  from  nearby 
structures,  power  and  telephone  lines,  etc.  The  antennas  could  be  placed 
in  more  desirable  sites  by  installing  two  broadband  preamplifiers  at  the 
base  of  the  crossed  loops  thus  permitting  the  use  of  longer  lead  lengths. 
This  would  overcome  some  of  the  coupling  errors  caused  by  placing  the 
antennas  in  close  proximity  to  electronics  shelters. 
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V.  CONCLUSIONS 


This  section  summarizes  the  evaluation  of  the  comparative  performance 
of  the  two  systems. 

1.  The  field  evaluation  of  the  ASC  system  at  the  time  of  this  test 
was  premature  since  the  device  was  still  under  development.  The 
system  satisfied  a  preliminary  performance  test  but  required  a 
continuing  effort  by  the  vendor  to  maintain  operation. 

2.  With  the  exception  of  the  telephone  modem  failure  and  the  clock 
restart  problem,  the  LLP  system  operated  on  a  24  hour  day, 
unattended  basis.  The  LLP  system  appeared  to  be  fully  developed 
and  operationally  ready  for  field  evaluation. 

3.  Based  upon  visual  observations  of  cloud-to-ground  lightning  events, 
the  LLP  system  demonstrated  a  superior  detection  rate  than  was 
observed  on  the  ASC  system,  viz.  84  percent  versus  56  percent. 

4.  Since  virtually  all  processing  in  the  ASC  system  is  done  at  the 
central  facility,  the  system  appears  to  be  limited  by  computational 
speed.  On  the  average,  location  estimates  are  produced  approxi¬ 
mately  five  seconds  slower  than  with  the  LLP  device.  Some  lightning 
events  may  be  missed  by  the  ASC  system  since  DF  data  acquisition 

is  interrupted  while  computer  processing  is  in  progress. 


VI.  RECOMMENDATION 


An  objective  evaluation  of  system  performance  cannot  be  accomplished 
while  system  modifications  are  being  incorporated  which  impact  device 
performance.  At  such  time  that  the  ASC  system  can  be  operated  in  an 
automatic  unattended  data  acquisition  mode,  a  side-by-side,  performance 
comparison  can  be  made.  If  this  condition  is  satisfied  and  sufficient 
interest  exists,  it  is  recommended  that  a  comparative  evaluation  be 
reinitiated. 
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